Boundary layers in stochastic thermodynamics 
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We study the problem of optimizing released heat or dissipated work in stochastic thermodynam- 
ics. In the overdamped limit these functionals have singular solutions, previously interpreted as 
protocol jumps. We show that a regularization, penalizing a properly defined acceleration, changes 
the jumps into boundary layers of finite width. We show that in the limit of vanishing boundary 
layer width no heat is dissipated in the boundary layer, while work can be done. We further give 
a new interpretation of the fact that the optimal protocols in the overdamped limit are given by 
optimal deterministic transport (Burgers equation). 
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With the advent of micromanipulation, thermody- 
namic quantities such as work and heat have taken new 
operational meaning for isolated microstates or single 
trajectories in phase space. The problem is then nat- 
urally posed to optimize such fluctuating quantities by 
varying the externally imposed conditions, usually called 
the protocol. The prime experimental system for such 
potentially optimized micromanipulation is particles or 
molecules in optical traps P, Q • Protocol optimization 
may also turn out to be important in improving novel 
computational schemes harnessing the advances of non- 
equilibrium statistical physics [3|. Many functionals of 
fluctuating paths may conceivably be optimized, but two 
of the most natural and important are obviously expected 
released heat to the environment, and expected dissi- 
pated work. For specific examples in stochastic thermo- 
dynamics, systems described by overdamped Langevin 
equations, Schmiedl and Seifert showed that the opti- 
mizing protocols have discontinuities Q . Several studies 
have tried to assign a physical meaning to such infinitely 
fast transformations, and even to look for an approximate 
process that would be amenable for real experiments [Hl- 
|7[. In a recent contribution using a Hamilton- Jacobi- 
Bellman approach we showed that the solutions of these 
examples are special cases of a more general scheme con- 
necting optimal protocols to optimal deterministic trans- 
port [81] . The discontinuities or jumps in the protocols are 
generic, and can be understood as the optimal determin- 
istic transport proceeding at constant speed from start 
to finish [8|. The infinitely fast transformations should 
be smoothened by inertial effects either in the system, or 
in a physical model of the protocol. 

In this Letter we show that a regularization by current 
acceleration (a concept to be defined) allows for equally 
explicit solutions to the problem and direct investiga- 
tions of the corresponding boundary layers. We hence 
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show from the limit of regularized solutions that no heat 
is released during the fast transformations. In this work 
we use extensively forward and backward derivatives of 
the stochastic process as developed for stochastic quan- 
tization . As a side effect we are thus also able to 
derive the earlier results on deterministic transport in an 
alternative way. 

The model we consider is the dynamics of the nonequi- 
librium transition of finite duration At = tf — to described 
by the Langevin equations in the overdamped limit 



d^t = -^dt + dwt , 



(1) 



with initial value — Xq, drift bt — d^^V{^f,t) and Wt 
a vector valued white noise with covariance {wtWt/) — 
6{t — t'), and mobility . is an M'^-valued stochastic 
process indexed by the open time interval I = [<o,if]- 
During the transition the control potential changes from 
V{x,to) = Ua{x) to V{x,ti) = U{{x) and the probability 
density p{x,t) evolves according to the Fokker-Planck 
equation 
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Following [11], an energy balance for the single stochas- 
tic trajectories of these dynamics yields the so-called 
stochastic thermodynamics. Defining the work done on 
the system during the time interval Ai as 



SW ^ J^' dtV{^t,t) dt, 



and the heat released by the system as 

'Ut^d^Ji^t.t) dt , 



(3) 
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then the balance dU = 6W — SQ resembles the first law of 
thermodynamics over the time interval [ta-,t{]. Note that 
the product in ^ must be defined in the Stratonovich 
sense. 

We now introduce the notions of the current veloc- 
ity and the osmotic velocity associated to the stochastic 
process Assuming that (HJ leads to a smooth diffu- 
sion process described by a transition probability density 
p{xt,t\yg, s), the mean forward derivative is defined as 



D^^ := lini / dx 
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The mean backward derivative can be written similarly 
using the opposite conditional probability p{$,f,t\x^t'). 
For Markov processes we can use Bayes' formula and 
write instead 

' t'ttj t-t' p{^„t) T ^ ' 

The mean forward and mean backward derivatives are 
related by 



(7) 



and the current velocity Vt and the osmotic velocity Ut 
are 



(8) 
(9) 
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while the mean forward (or mean backward) derivative 
by itself has a diffusive term, in the symmetric derivative 
of Eq. [10] it cancels out. Correspondingly, the Fokker- 
Planck equation is always deterministic mass transport 
in terms of the current velocity 



Vt ^ (bt + b,t)/2T , 
Ut^{\/PT)di, lnp(^„<) 

For any smooth function /(^j) wc have 



(11) 



We now use the current velocity and osmotic velocity in 
the heat and work functionals over the interval I, which 
we define as the expectation values W = E SW and Q — 
E 5Q respectively. Straightforward application of the ltd 
lemma (see e.g.[12|) yields the heat functional 



Q = E 
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If the probability measure p decays sufficiently fast after 
an integration by parts we can write 



Q-E/ dt[\\vtf+UfVt] 



(13) 



Probability conservation and the definition of u then 
yield 



Q^^Eln^ 

/3 Pto 
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(14) 



From this follows immediately an inequality for the work: 

W>E\ui-Uo + ^\n^\=Ff-Fo = J^, (15) 

which is a form of the second law of thermodynamics. 

In our earlier contribution [8] the control was the drift 
b, and the functional was ([T^ . Proceeding as above we 
can take the control to be v, and the functional to be 
(HH) . Given that ([TU|) and pT|) are already inviscid equa- 
tions this means that we can directly interpret ([14]) as a 
deterministic optimization problem the solution of which 
must be an inviscid equation (diffusion does not appear). 
To find that inviscid equation, which is Burgers equa- 
tion [i|, = dxi^/r, 
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explicit calculations equivalent to those in 
performed. From (|16p it follows that 

E r dtT\\ Vt ip=2E r dt^ 

Jt„ Jt„ dt 



(16) 
must be 
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implying for the heat released during the optimal trans- 
formation the expression Q 
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with p evolving according to pT|) . In the spe- 
cial case of Gaussian initial and final densities, 
pix,to) = (/3/27r)'^/2e-('5|l^ll'/2) and p{x,t{) = 
(;3/2 7rcr2)rf/2e-(/3il^-''llV2T^) ^ith h a constant vector, 
the heat released by the optimal protocol over an a time 
horizon At is 
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and the optimal current velocity 

{a-l)x- 



V* {x,t) = 
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At at 



(19) 
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with at — [{tf — t) + (t — to)a]/At a linear function of 
t. A surprising property of this optimal driving (first 
obtained in [1] for the minimization of the work ([3])), 
is the existence of discontinuities at the initial and final 
times of the transformation. 

We will now turn to the main topic of this Letter, which 
is to regularize the optimization by penalizing the current 
acceleration 
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FIG. 1. (Color online) Average position Xt, velocity it, nor- 
malized acceleration ^/e xt as obtained from (|29|1 . and aver- 
aged action A, for a time interval [0, 1] t = h — a = 1, 
and different values of e: 0.1 (black curve), 0.01 (red curve), 
0.001 (green curve) and 0.00001 (blue curve). In the bottom- 
right panel the dashed line corresponds to the overdamped 
value, e = 0. 



We note that v and a are as rough functions as ^ along 
trajectories (but no rougher). The current acceleration 
would be a complicated expression in terms of the original 
drift field b and density field, but the heat functional reg- 
ularized by current acceleration preserves the same time 
symmetry as the heat functional itself. With these pre- 
liminaries, the problem of determining the minimal heat 
released in a transformation between given states reached 
with assigned values of the initial and final current ve- 
locity reduces to the problem of finding the minimum of 
the functional 



A-.^'E dtT{\\ vt IP +eT^ 
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In (1^^ the Lagrange multiplier A enforces the constraint 

Xtj = 4>{xo) , with Xt^ = Xo and Xt — Vt . (23) 

and the map </> specifies the relation between the initial 
and final states 
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(24) 



By (O, ([H]) we can write the initial current velocity as 

(25) 



— - Inpt^ (x) 

T pr 



and similarly for Vt^ . It follows that if the current velocity 
vanishes at the boundary of the control horizon. 







(26) 



the initial and final probability densities correspond to 
equilibrium states. Furthermore, if the initial an final 
states are Gaussian, as used above to obtain (fT9|) . then 
the boundary conditions (f23| reduce to 



, and Xo = {xtj — h)/a 



(27) 



In general finding the map is the main obstacle hinder- 
ing the derivation of explicit solutions. If we, however, 
consider the initial state and the map as boundary in- 
put we can recast the optimization problem into the sim- 
pler problem of minimizing the action of a classical un- 
stable oscillator in a shifted potential t \\ y— A/(2t) |p. 
The identifications = Xt = Vt provide the connection 
to the original problem and the boundary conditions (pS)) 
and (|27)) . From the stationarity condition for Eq. (|22l) . 
we obtain the Euler-Lagrange equation 



2r (er^ 



Xt-Xt] 



(28) 



whence for the boundary conditions (|23p . ([26)) it follows 

( 
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with 



2t [h- 



1) Xo] 



At - 2t y/etanh 



(29) 
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The average position Xt and acceleration Xt are obtained 
from (|29p . The convergence of the regularized solution 
toward the overdamped case of [§| is shown in Fig.[T] Fur- 
thermore, expressing the action functional (|22|) in terms 
of the stationary solution and averaging over the initial 
state we obtain 
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Ai,{h,(7,e) = 



T II h 11^ +d {a-iy (3 
A<-2rV^tanh(^) 



(31) 



It is straightforward to verify that in the limit of vanish- 
ing e, E ln(pf J / pt^ )//3+Ai, reduces to the overdamped re- 
sult (see bottom-right panel of Fig. [IJ . Furthermore, 
for any small but finite e our regularization unambigu- 
ously determines through ([8]) , ^ the control potential V 
{bt = d^^V) in the closed control interval I. This means 
that for any e > the optimal work expression 



W^^E{Ui-Uo + In 



Pta 



-A 



(32) 



is well defined. In particular, for transformations between 
equilibrium Gaussian sates we have immediately = 

Finally, we consider the minimization of (|22|) under the 
hypothesis that the final state is still Gaussian but out of 
equilibrium. In particular, we suppose the final value of 



4 




0.2 0.4 0.6 0.8 1 
t 



0.2 0.4 0.6 0.8 1 
t 



0.2 0.4 0.6 0.8 1 



FIG. 2. (Color online) Average position xt, velocity xt, nor- 
malized acceleration ~Jext as obtained from (|33|l . for a time 
interval [0, 1], x — —0.1, a = 3.5, t = h = 1, and ytf — 1 
(upper panels), j/tj = 1.5 (lower panels). The different curves 
are for e: 0.1 (black curve), 0.01 (red curve), 0.001 (green 
curve) and 0.00001 (blue curve). 



the control potential U{ = c\x — h\'^ /2 to differ from the 

osmotic (equilibrium) potential ln[p/ (2 tt (t^//3)'^^^] = 
— |a; — /xp/2CT^, thus implying a non- vanishing final cur- 
rent velocity. Proceeding as before we obtain 



Xt — Gx sinh 



X 

27 



cosh 



- 1 



,(33) 



with G\ = [2ri/(j — A(ca — l)]/2rsA and where, to ease 
notation, sx — sinh {At / y/er) , cx = cosh (Ai/y^r), and 



A = 



CA + lytfV^-r(cA-l) -{fl + Xo (o--l))sA 



SA 



(At/2r)(cA + l)- V^SA 



.(34) 



In the limit of vanishing regularization the minimal work 
done on the system to operate the transformation tends 
to 

>V.-4°(£ff + -"7^"^ )-fQ.(;.,a) (35) 

whilst within the open interval {to,t[) the mean state of 
the system changes linearly as Xt = x + {t — to)[fi + x {a — 
1)]/At independently of the final value of the current 
velocity y^j:. We illustrate this phenomenon in Fig. [21 

From (j35p we can determine the Gaussian nonequi- 
librium state which, given the final value of the control 
potential Uf, minimizes the work. A straightforward cal- 
culation shows that the minimum is attained for /x = 
cAth/{cAt + 2t) and cr'^ ^ [y^ At {c At + 2 t) + t'^ - 
T]V(At)2. Thus, we recover the result of 0, i] for the 
minimal work transforming an initial equilibrium state 
under the constraint that the protocol at the end of the 
control horizon should attain an assigned final value. Our 
regularization framework allows us to interpret such work 
as lower bound over the work done between given states 
positing that it is possible to retain knowledge of the final 



protocol but the knowledge on the final non-equilibrium 
state is lost. 

In summary, we have investigated optimal control in 
stochastic thermodynamics. First, we have shown that 
the optimal control equations for heat and work transfor- 
mations between given states have a natural interpreta- 
tion in terms of functionals definite under time reversal of 
the Markov process describing the overdamped dynam- 
ics. Second, we have proposed a regularization frame- 
work in terms of current acceleration. The regularization 
allows us to identify without ambiguities the internal en- 
ergy of the system with the drift potential. In the limit 
of vanishing regularization, the current acceleration tends 
to zero within the control horizon but diverges (as e^^/^ 
in the examples considered) at the control-horizon end- 
times thus carrying no contribution to the heat release. 
Correspondingly, the optimal protocol converges toward 
the overdamped solution by forming boundary layers i.e. 
regions of faster variation at the control horizon bound- 
aries. As e vanishes, these regions shrink to measure zero 
sets over which the internal energy forms in the limit 
discontinuities bringing finite contributions to the work 
done on the system during the transformation. In con- 
clusion we achieved a fully-consistent theoretical picture 
of optimal overdamped thermodynamics well suited for 
the interpretation of experimental and numerical data. 
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